Deploying a Linear Regression Algorithm

How to take the output of a linear regression algorithm and deploy it into a production environment

Table of contents

  1. The Problem
  2. Loading the Data
  3. Re-building the Model
  4. Reviewing the Linear Regression Algorithm
  5. Deploying the Linear Regression Algorithm
  6. Conclusion

Photo by Dlanor S on Unsplash

The Problem

A little while ago a colleague from a company I was working with took a keen interest in one of my data science projects which was developing a linear regression to make probabilistic predictions about future cases of student retention (i.e. when students are likely to complete their courses or when they are likely to drop out before completing).

My colleague asked me how the result of all this data science work could be extracted out of the Python Machine Learning algorithm and deployed into the production environment (which is not written in Python) so that users of the core production systems could spot students who are likely to drop out and then give them additional support to help them succeed.

That got me thinking about how I could extract the machine learning algorithm from Python and encode it in a different programming language (for example Excel VB Macros, C#.NET etc.) which led to this article.

As the student data is private I used a public data set from Kaggle relating to the prediction of the sales prices of houses. The full Kaggle data set can be found here

Loading the Data

Let's start by importing the libraries we will need for the rest of the Notebook and by reading in the data which has been downloaded from Kaggle and placed in a sub directory called data.

Note that the data has already been through the data science pipeline. It has been cleaned, explored, modelled, tested, validated and interpreted and these steps are outside the scope of this article.

We are assuming that the data and the model are sound and that we just want to transfer the machine learning algorithm into a production environment.

Here is what our final data set looks like -

To avoid making the data and hence the article over-complicated I have chosen to drop some of the features that the previous stages of the data science pipeline had concluded were only making a marginal contribution to the prediction algorithm -

The final data set we will be working with is as follows -

Although the modelling stage is out of scope for this article, I did think it would be worth expending 1 line of code to have a look at the shape of the data. The following pairplot shows how each feature is distributed and confirms that the remaining features are correlated with the sale price which is our target / dependent variable.

Re-building the Model

The machine learning model was build in a previous stage of the data science pipeline but it is going to be re-built here as we need the LinearRegression object to build our deployed algorithm.

Photo by Charles Deluvio on Unsplash

Here we are splitting the data into two sets, 70% of which will be used to train the model and 30% to test the model and then fitting the linear regression model to the training data set ...

Again, the accuracy of the model would have been established in the "Test and Validate Model" stage of the data science pipeline but let's just have a quick look to make sure the model is performing as expected ...

Note: R-squared is a statistical measure of how close the actual data values of y are to the fitted regression line and in our case the model gets an R-squared of 74.6%.

However R-squared suffers from a disadvantage: the more features are added, the higher the value of R-squared. Hence the "Adjusted R-squared" is a better measure as it takes account of the number of features and just adding additional features can reduce the adjusted R-squared value.

Our Adjusted R-squared is 74.5% and this is good enough for out purposes, so let's continue.

The next line of code generates the predicted sales prices using the model and stores the results in y_pred

Now let's take a copy of the testing data, add the actual price back in and also add in the predicted price that was generated by our linear regrssion machine learning algorithm ...

It all looks as expected. Now we can move onto the main objective which is to find a way to "productionize" the algorithm and to deploy it.

Before we do that, let's quickly review the linear regression algorithm and what it means ...

Reviewing the Linear Regression Algorithm

A linear regression generates a straight line that best fits the independent variable (X axis) and dependent variable (y axis) -

Image by author

In a linear relationship between two variables like the one in the image the value of y can be predicted using a linear equation as follows -

$$y = mx + b$$

Here $m$ is the co-efficient of $x$ and $b$ is the intercept.

Our model has 4 features and it can be represented as a linear equation with 4 co-efficients as follows -

$$y = ma + nb+ oc + pd + e$$

... where a, b, c and d are our data values, m, n, o and p are the "co-efficents" we must multiply them by to predict the answer and e is the intercept.

Once the model has been built the LinearEquation class can be used to retrieve the co-efficients and the intercept of our linear equation as follows -

If that is still a bit difficult to visualise the following Python function will create a simple DataFrame from the output of the model that we can use to easily see and use the output of the algorithm -

Note that I have included the standard deviation because if you want to compare the co-efficients they must be multiplied by the standard deviation to be expressed in the same units as the following example illustrates -

Image by author

Deploying the Linear Regression Algorithm

Now we have everything ready and we can deploy the linear regression represented by our model in a few easy steps.

First let's have another quick look at the test data including the actual price and the predicted price ...

.. and also quickly review our co-efficient and intercept values -

Now let's have a look at the values of the 4 features for the top row in our test data ...

To "productionize" our linear regression algorithm all we have to do is multiply the value of each feature in our chosen row of data with its associated co-efficent and add the intercept value as follows -

This shows that the sales price for the row with index 1164 was predicted to be 162748.547404 by the linear regression machine learning model and the amount predicted by our simple linear equation which used the co-efficent and intercept values was 162748.54761699997.

Our simple linear algorithm matched the model output to 3 decimal places which I have assumed is a rounding issue and it is certainly close enough to implement in our production environment.

Conclusion

Now that we know the co-efficients for each of the features and the intercept value we can write a very simple function that could easily be ported to Excel VBA, C#.NET or any other programming language.

This function is deceptively simple because it is the culmination of the entire data science pipeline of activities consolidated into one simple function, but in a real world data-science project there will have been a lot of work invested in getting to this point.

Please note that if this were a real-world example I would periodically re-run the data science pipeline on current data to ensure that my chosen features, their co-efficents and the intercept are still a good representation of the real world and that they are producing predictions that are sufficiently accurate to use, or alternatively I would update the co-efficient and intercept values.

Here is the final function, written in Python, but easily ported into any other language -